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We study the spacial and temporal multiscale properties of complex systems. We present acceler- 
ated algorithms for dilute spin glasses and display explicitly their relation to the effective dynamics 
of specific collective degrees of freedom (macros). 

We discuss the difficulties in applying multiscale-cluster algorithms (MCA) to general frustrated 
systems: MCA does not succeed to break the system into clusters. We relate these difficulties 
to rigorous negative results in systems with an ultrametric space of ground states: the tunneling 
between vacua cannot be expressed into an algorithm acting upon independent macros. 



I. COMPLEX SYSTEMS AND MACROS 

One of the main characteristics of complex systems 
is their computational difficulty: the time necessary for 
their investigation and/or simulation grows very fast with 
their size [1]. The systematic classification of the the diffi- 
culty and complexity of computational tasks is a classical 
problem in computer science [2]. 

In physical systems, the emergence of large time scales 
is often related to multiscale spacial structures within 
the system. Long range and long times scale hierarchies 
(Multiscale Slowing Down) are usually related to col- 
lective degrees of freedom (macros) characterizing the 
effective dynamics at each scale. 

The physical understanding of complex macroscopic 
phenomena is then often expressed through identifying 
the relevant macros and their effective dynamics (e.g. 
hadrons in the theory of quarks and gluons, Cooper pairs 
in superconductors, phonons in crystals, vortices in su- 
perfluids, flux tubes, instantons, solitons and monopoles 
in gauge theories, etc.). 

One can entertain the hope that many complex sys- 
tems display some kind of universal multiscaling expo- 
nents generalizing the scaling critical exponents of the 
critical systems. One could hope for the existence of 
some kind of Multiscale-universality classes generalizing 
the universality classes of Renormalization Group theory. 
Such a situation would have a significant unifying effect 
on a very wide range of phenomena spreading over most 
of the contemporary scientific fields. 

In the absence of a rigorous theoretical basis for such 
a hope, its investigation relies for the moment mainly on 
the use of computers. In particular one uses "first prin- 
ciples" simulations which implement directly and with- 
out the intermediary of ad-hoc approximations the fun- 
damental physics of the systems under study. 

Usually, it is the dynamics of the macros during simu- 
lations which produces the Multiscale Slowing Down and 
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1 see for example of the projection by PTMG of exact lattice 



reciprocally, the slow modes of the simulation dynamics 
project out the relevant macros [12]. 

Therefore, a better theoretical understanding of the 
multiscale structure of the system, enables one to con- 
struct better algorithms by acting directly on the relevant 
macros. Reciprocally, understanding the success of a cer- 
tain algorithm yields a deeper knowledge of the relevant 
degrees of freedom of the system 1 . 

The present paper implements this point of view into 
the study of spin glasses. 

Section 2 introduces the basic notions of Multiscale- 
Cluster Algorithms (MCA). 

Section 3 describes the difficulties in applying MCA to 
generic frustrated systems. 

Section 4 contains rigorous results which forbid macros 
in ultrametric (UM) systems. 

Section 5 identifies the relevant macros and their role 
in constructing MCA for Dilute Spin Glasses. Section 6 
demonstrates numerically the efficiency of the resulting 
MCA. 

Section 7 summarizes the conclusions. Appendix A con- 
tains the proofs of the results stated in Section 4. 

We interpret the negative results in the pair of sections 
3-4 and the positive results in the sections 5-6 as support- 
ing in both directions the relation between macros and 
the efficiency of MCA. 



II. MULTISCALE-CLUSTER ALGORITHMS 

(MCA) 

An example of multiscale effective dynamics and its 
related multiscale slowing down is the critical slowing 
down (CSD) at second order phase transitions. There, 
the relaxation time r diverges with the systems size L as: 

t ~ L z 



Atyiah-Singer modes [8]. 
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where z (~ 2) is the dynamical critical exponent. Conse- 
quently, the typical time needed to produce a large Boltz- 
mann set of decorrelated configurations diverges and the 
standard local Monte-Carlo methods become inefficient. 

It was shown that when the detailed knowledge on the 
relevant macros is included in the simulation algorithms, 
the value of z can be reduced dramatically (down to 0) 
[28]. These algorithms, which we will call generically 
here Multiscale-Cluster Algorithms (MCA), allow very 
fast and precise computation of the equilibrium thermo- 
dynamic properties of the systems. However their main 
importance is to guide and validate by objective means 
(lowering of z) the intuitive guesses on the physically rel- 
evant macros and their macroscopic dynamics [12]. 

We are treating the various (MCA) in a conceptually 
unified way: as expressions of the macros appearing at 
various scales. In fact many of the explanations in Sec- 
tion 5 on the dynamical relevance of the spatial struc- 
tures manipulated directly by the Macros Reduction Al- 
gorithm (MRA) can be given equally in the language of 
Dynamical Algebraic Multigrid (DAMG) [9] as well as in 
the framework of the Cluster Algorithm (CA) [5]. 

In statistical mechanics systems, the objective of MCA 
is to generate as fast as possible a representative sample 
of configurations. This is realized by acting directly on 
the macroscopically relevant macros (in contrast with the 
usual local algorithms which act on the microscopic ele- 
mentary degrees of freedom). 

The typical cluster algorithm (CA) works according 
the following general principles: 

1. One selects a particular subset of allowed changes 
for the degrees of freedom associated with each site 
i = 1 , . . , N of the system 2 . 

This reduces the system to an Ising-like system 
S = {si,...,sjv} where the Ising variables sj, can 
take the values ±1. A configuration is a specific 
assignment of one of these values for each s^. The 
interaction energy 

E(S) = ^J ilj (l-s i s j ). 

is parametrized by the link parameters Ji j associ- 
ated with each pair of sites i and j 3 . 

2. One constructs and updates a system of clusters 
which preserves the macroscopic dynamical prop- 
erties of the initial spin system. 



The system of clusters and its dynamics is obtained 
by modifying the (link) parameters Ji j between 
the pairs of spins (i, j). More precisely the link (i, j) 
is either "frozen" Jij = oo or "deleted" Jij = 
based on the following classifications. 

Consider the current values s f t and sf of the 2 
spins and their current energy: E(Sfj) = ^J, : j(l — 

*?*?)■ 

• If the spins are in the low energy state 
E(Sfj) = 0. the link is called saturated 
or satisfied. Otherwise, the link is said un- 
satisfied. 

• If the difference between the satisfied and un- 
satisfied energies of the link \ Jij\ is big, the 
link is called strong. Otherwise it is weak. 
Often the strong-weak label is given relative to 
the actual temperature of the system. For in- 
stance at very large temperature (T >> |J/j|) 
all the links can be considered weak while at 
low temperature (T << |=//j)|) most of the 
links may be acting as strong. 

With this terminology, the cluster generating pro- 
cedure is: 

(a) freeze (with high probability) the strong sat- 
isfied links (links with low energy). 

(b) delete the strong unsatisfied links (links with 
high energy). 

(c) give for weak links an appropriate stochas- 
tic chance to both options (frozen-deletion) to 
arise. 

(d) flip the relative signs of spins which belong (by 
the link deletions) to different clusters. 

In the sequel we will call loop a closed chain of links 

{(ik, ik+i) \ k = I, n and i n+1 = ii}. 

If the product 

n Ji * .•■*+! < ° 

k = l,n 

is negative the loop is said frustrated. If a loop is frus- 
trated there exists no spin configuration for which all the 
links of the loop are satisfied. 



2 E.g. in finite temperature SU(2) gauge theory, the SU(2) matrix degree of freedom on the time-like links are allowed to 
change only their signs during a MCA step [10]. This is an algorithmic expression of the physical understanding that it is the 
center of the group which is the relevant degree of freedom. 

3 For notational convenience, we use in this and the following section a definition of the total energy which differs by an overall 
additional constant Eo = i ^\ J ltJ from the definition used in the rest of the paper. 
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III. MCA DIFFICULTIES IN FRUSTRATED 
SYSTEMS 

The problem of the applicability of MCA to frustrated 
systems arised quite early since most of the cases in which 
the MCA did not work were situations in which the first 
step of section 2 reduced the system to a frustrated one 
[3,12] 

Some of the most important families of frustrated sys- 
tems are the Randomly Frustrated Systems (RFS) and 
the Spin Glasses (SG). 

A typical SG system presents a complex energy land- 
scape consisting of many local minima, separated by huge 
barriers which scale with the size of the system. This is 
expressed by the emergence of an ultrametric structure 
of the ground states space and an infinite hierarchy of 
exponentially divergent relaxation times [13]. 

To understand the difficulties which occur when ap- 
plying MCA to frustrated systems let us analyse in detail 
a simple scenario. Suppose that the configuration (CI) 
in the following figure: 



A 




unfrustrated chain, 
frustrated chain. 



is an energy ground state (GS). Consider the case that 
the subsystems X and Y are linked by 2 chains of links 
A and B (as shown in the figure). 

Suppose that in CI, the total energy of the links be- 
longing to A is E\ = while the energy of the chain B 
is is Eg = e. The chain B is therefore frustrated while 
the chain A is unfrustrated. For the simplicity of the 
argument let as assume that the system is at very low 
temperature, T << e though the conclusion is valid at 
higher temperatures (T < e) as well. 

Assume now that a new ground state (GS) configura- 
tion C2 is obtained by flipping in CI the relative sign 
of X and Y such that E 2 B = and E\ = E B = e. This 
means that in the ground state C2 it is the chain A which 
is frustrated while B is unfrustrated. The total energy of 
C2 is equal to the total energy of CI. An efficient algo- 
rithm should allow one to easily obtain C2 from CI and 
viceversa. 

However, with the usual cluster algorithm, this condi- 
tion is not fulfilled. Indeed, if the chain A is frustrated 
(in C2) so that the chain eventually would be cut by the 
Cluster Algorithm (CA) then, automatically B is unfrus- 
trated and all its links will be eventually frozen by the 
CA. Vice-versa: if B is cut, A is uncut (this happens 
in CI). Consequently, in any case, at least one of the 
chains A and B is frozen. Therefore X and Y are always 



included by the CA in the same cluster and there is no 
way to get from CI to C2. 

Therefore it appears that a frustrated disordered sys- 
tem is not compatible with an efficient updating algo- 
rithm. This argument is getting even stronger if X and 
Y are linked by several chains. We will present in section 
4 some rigorous results extending these intuitions to a 
wider class of systems. 

In a few special cases one can overcome these prob- 
lems. If the problem is local and the structure of the links 
is completely known, then a "two bond deletion" might 
help such as in the fully frustrated system on a square 
lattice [14]. The "two bond deletion" technique can be 
extended in other systems to 3 and 4 bond deletions [24] 
and even to an "n"-bond procedure. However, in general, 
the problem would still revert to exponentially combina- 
torial complexity if one has no a-priory knowledge about 
which "special" subset of links one should combine. 

The simulated annealing technique which helped some 
systems from getting stuck in local minima, failed to pro- 
vide a complete solution in SG case. A related direction 
is the SW replica algorithm [5] and its modifications [23] 
which use simultaneously various replica of the system 
in order to identify large spatial regions which act co- 
herently. This might work in few simple cases, in low 
dimension (until now only 2D). However, one cannot ex- 
pect such an algorithm to work for a general frustrated 
system because: 

• in general (e.g. for spin glasses [4]) one can find 
for each GS exponentially many metastable states 
with energy close to the ground-states. As a re- 
sult one needs in general an exponential number of 
replica in order to fully capture the structure of the 
system. 

• The spacial regions which have to flip in order to 
turn various ground-states one into another cannot 
in general be identified and manipulated as inde- 
pendent entities (see in Section 4 the relevant the- 
orems for ultrametric systems). 

These limitations imply that one cannot get away from 
the combinatorial complexity of the general randomly 
frustrated problem. The CA logic of constructing clus- 
ters, is based on local (see e.g. [27]) features imply- 
ing local criteria such as deleting the unsatisfied links 
which define the block boundaries in the ferromagnet 
case. However in a general case the feature of the cluster 
is only apparent in a global view, without any local signs 
(see theorem III). 

The situation can be compared with having to find 
one's way in a labyrinth in the phase space: each small 
local change in the position of the potential energy 
labyrinth walls determines large unpredictable changes of 
the solution route depending on details scattered across 
the entire phase space. 

Consequently, we are discerning 3 main complexity 
cases: 
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• In very simple cases the pattern of complexity is 
reducible (may be by an iterative multiscale proce- 
dure) and the MCA's capturing this reducible com- 
plexity are an efficient computational and concep- 
tual tool. 

• In the general case one has to put an exponen- 
tial computational effort to fully "understand" the 
structure of the system. This situation is sim- 
ilar with "understanding" the architecture of a 
labyrinth and is expressed by the theorems of Sec- 
tion 4. 

• In some cases the system contains certain macros 
which are "irreducibly complex". Yet the interac- 
tions between these macros are tractable by MCA 
or other algorithms. In these cases, MCA can help 
reduce the "less complex" part of the dynamics 
leaving the "irreducible core" for a separate treat- 
ment. 

The last possibility has been exploited in the Parallel 
Transported Multigrid (PTMG) [11,8] treatment of the 
fermions in gauge field background where the complexity 
related to the gauge freedom was eliminated at the MG 
level while for macros related to frustration and topol- 
ogy (e.g. Atyiah-Singer zero-modes) one has developed a 
method [12] for implicit identification, manipulation and 
elimination of irreducibly complex macros. Similar in- 
tuitions are at the basis of the successful algorithms for 
diluted spin glasses described in sections 5 and 6. 

Recognizing the "irreducibly complex" parts of a com- 
plex system (rather than trying vainly to solve them by 
multiscale means) might be a very important aspect both 
conceptually and computationally. 

IV. ULTRAMETRIC (UM) SYSTEMS DO NOT 
HAVE INDEPENDENT MACROS 

As explained above, the SG systems present a certain 
hierarchy in their energy landscape which is responsible 
for the hierarchy of time scales characterizing their multi- 
scale slowing down. This rugged energy landscape is also 
the origin of the ultrametric properties of their ground 
states space. 

One could hope to make some relation between the 
SG ultrametric hierarchy and the existence of an effec- 
tive representations of the dynamics in terms of a spacial 
multiscale hierarchy of independent macros. This in turn 
would become the basis of an efficient MCA. 

It turns out that the case is exactly the opposite: 
the ultrametric hierarchy characterizing SG insures the 
inexistence of a representation of the effective macro- 
scopic dynamics of the complex system in terms of their 
macroscopic disjoint sub-sets (i.e. a complex ultramet- 
ric system is not effectively reducible to a set of sub- 
systems). 

Let us prove it in detail. 



Consider an Ising-like system with variables Sk = 
±1, k = 1, ...,N. The energy of a configuration S c = 
{s?,...,s%} is E(S C ) = -i£ 8J J hJ sfsf. The ground 
states (GS) of S are the configurations with the lowest 
energy density (states differing only by a finite energy 
(0(7V°)) are considered degenerate). 

The metric in the configurations space is defined by the 
following distance. If 2 configurations S° and S A differ 
(only) by the sign of the spins belonging to a subset A 
then their distance is 

d(S° , S A ) = #A = the number of elements in A. (1) 

Configurations differing only by a global change of sign 
are considered identical. This brings the support of the 
function d(S A ,S B ) into the interval [0,0.57V]. 

We shall call a system ultrametric (UM) if the 
GS's with the above metric form an ultrametric space. 
Namely, for any 3 GS's S c , S A , S B one has 

d(S A ,S B ) < max[d(S c ,S A ),d(S c ,S B )}. (2) 

Note that for real systems this condition is fulfilled, prob- 
ably up to measure zero of violations, and up to some 
small e ~ d/N [18]. Those limitations are not affecting 
our final conclusions though one should be aware to their 
existence. 

The first theorem expresses the fact that one cannot 
hope to travel between various GS's of a ultrametric sys- 
tem by just identifying and flipping independently vari- 
ous subsets (macros/collective objects). 

More precisely: 

Theorem I: 

• Consider 2 subsets A and B of of a ultrametric sys- 
tem S which has a GS S° . Assume that the states S A 
and S B obtained by flipping (only) the spins of the set 
A (respectively B) in S° are GS's too. 

then 

• For #A = #B 

#(Af]B)>l/2#A (3) 

• For #A ^ #5 

#(Af]B) = l/2min(#A,#B) (4) 

Theorem I means that at least half of the spins of one 
of the sets (A or B) belong to the other set. This is 
hardly ones idea of two independent sets. Moreover, gen- 
erally, a point belongs to an infinite number of strongly 
overlapping clusters. This implies that locally one has 
no criterion for constructing the relevant macros. Those 
can be identified only from a global view. 

In conclusion, in ultrametric systems it is ruled out 
that various regions of the system can be treated as in- 
dependent collective degrees of freedom (macros). This 
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picture can be extended to finite but small temperatures 
with the help of the "pure state" concept [1]. 

This failure of separability of the whole into (almost) 
independent parts has conceptual implications in the 
sense that one cannot "understand" the complex system 
by "analyzing" it into its parts. In this sense an ultramet- 
ric system is conceptually irreducible to simpler entities. 
We will see in the next section that optimal global algo- 
rithms reduce in fact a system to its "irreducible" core. 

One is tempted to conclude that the entire discussion 
of reductionism can be reformulated in terms of "irre- 
ducible complex systems". I.e. in place of assuming ul- 
trametricity and deducing the inexistence of independent 
dynamical sub-objects, one can propose this dynamical 
inseparability as the fundamental property underlying 
irreducible complexity. 

Theorem I suggests therefore that one should engage 
in the systematic study of the systems which have fami- 
lies of GS's differing by strongly overlapping subsets. The 
topology induced in the system by these subsets might 
have interesting properties. 

In the sequel of this section, we will give further char- 
acterization of the sets A which relate (by their flipping) 
different ground states. 

Theorem II: 

• If S° and S A differing (only) by the sign of the spins 
in the region A are both GS of a (not necessarily ultra- 
metric) system S, 

then 

• their actual spin arrangement restricted to the sys- 
tem A alone is a GS of the system A considered as isolated 
from the rest of S. 

Note that this statement holds not only in low di- 
mensions (where the surface energy is not extensive). To 
see in which respect this statement is non-trivial, note 
that in the presence of the system A (the complement 
of A in S), the spins in the system A are submited to 
the influence of the external (to A) action of the spins in 
A. For a general subset A of S, this will bring the spins 
of A into positions which are not necessarily optimal in 
terms of the internal A interactions alone. They would 
be in general in a position which strikes a compromise 
between minimizing the internal A energy and the inter- 
actions with the rest of the system (A). Theorem II finds 
conditions in which the action of A can be ignored. 

This property has interesting uniqueness consequences 
on the sets flipping between GS's of ultrametric systems. 

Theorem III: 

• If S° and S A differing by (and only by) the signs of 
the spins in A are GS's in the ultrametric system S. 

then 

• their actual spin arrangement restricted to the sys- 
tem A alone, is the unique GS of A considered as 
isolated from the rest of S. 



This theorem throws some ironical light on the prop- 
erties of GS's in ultrametric systems: a posteriori there 
is something qualitatively special in the sets which con- 
nect between GS's of UM systems: the uniqueness of the 
their GS. These sets are very special and do not share 
at all the proliferation of vacua characteristic to typical 
SG systems. In fact theses subsets are not UM systems 
by themselves: Theorem III implies (among other things 
that) these subsets cannot constitute a hierarchy of UM 
sub-systems included recurrently one into the other. 



V. MACROS REDUCTION ALGORITHM (MRA) 
IN DILUTE SPIN GLASSES 

The "no-go" arguments of section 3 had a rather de- 
pressing effect on the expectations of the practitioners in 
the field on the performance of MCA in frustrated sys- 
tems. We will see below that these arguments and even 
the theorems presented in section 4 still allow for a signif- 
icant contribution of MCA in frustrated systems as long 
as they possess macros. 

As opposed to fully connected models such as SK [21], 
the geometry of the diluted models includes topological 
structures capable to engender such macros. The cluster 
algorithms (CA) can then locate and act on large regions 
of the configuration which are weakly linked to the rest of 
it. In addition to CA we construct a Macros Reduction 
Algorithm (MRA) which acts explicitly on the same 
macros on which CA acts stochastically. In this way we 
make explicit the role of the macros in both algorithms. 
The MRA has a structure very similar to the Dynamical 
Algebraic MultiGrid (DAMG) of [9]. Since its action is 
more direct, MRA is more efficient than CA in the mod- 
els for which it was designed (see Section 6 for numerical 
details). However CA is more versatile. 

We consider again the Ising-like system 

H = —^.JijSiSj (5) 

with the probability distribution for Jij: 

P(J tj ) = (1 - c/NWJn) + (c/#)/(J 8J ). (6) 

f(Jij) is the distribution of the surviving links after the 
dilution. 

This model, is known as the highly diluted system SG 
with finite average connectivity c(= 0(1)). The proba- 
bility for a spin to have connectivity k in such a system 
follows the Poisson distribution: c k exp(— c)/k\. 

Many geometric properties of this system are well un- 
derstood [19,20]. In particular, the system undergoes 
a percolation transition at c = 1. The maximal clus- 
ter is of order O(logN) for c < 1, 0(# 2/3 ) at c = 1, 
and O(N) for c > 1 where its size is explicitly given by 
P=l-P = l- exp(-c_P). 
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The finite connectivity models at low temperature are 
directly connected to the graph partitioning problem 
[19,20] (dividing a graph into subgraphs, with minimum 
connections between them). 



A. The Macro Reduction Algorithm 

In the introduction, it was claimed that the very ex- 
istence of an efficient MCA may help identify relevant 
macros in the system. 

To achieve this, we construct a Macros Reduction Al- 
gorithm (MRA) which freezes explicitly spins into macros 
and by doing so reproduces (and improves over) the per- 
formance of the efficient Cluster Algorithm (CA). 

MRA consists of the following iterative steps 

1. Access the points i of the system iteratively starting 
preferably with the ones with lower connectivity. 

2. For an accessed point i find its strongest connection 
Jij defined by |J 8J | > \ J iik \ , for all k ^ j. 

3. Freeze and Sj into a macro such that —SiJijSj 
is minimal. From now on the value of Sj alone la- 
bels the state of the macro and is just a "slave" 
determined by it. 

4. Ignore in the subsequent up dating 's of the sys- 
tem (in particular in the updating's of Sj). 

5. As seen in the detailed explanations below, when 
all points (macros) with 1 neighbour are exhausted, 
all trees are shrink to points. 

6. When all points (macros) with 2 neighbours are 
exhausted, all linear chains in the system shrink to 
length 1. 

7. After that, some regular MC (either local or global) 
which acts on the new system can be used. 

Note the the "reduction" stage takes negligible time (few 
MC steps) compared to the relaxation part!. 

The construction of macros in MRA is very similar to 
the block construction in Dynamical Algebraic MultiGrid 
(DAMG) described in [9]. 

Let us see now in detail why MCA's such as CA, AMG 
and MRA, work where the local algorithm doesn't. 



B. trees 
Consider a configuration: 




Assume X is minimal if = +. Then the energy is 
minimized if all s's on the tree sites are +. However , a 
simple MC algorithm might have problems in reaching 
this minimal energy configuration. For instance if one 
has: 




then even after insuring = +1, Sj will have 2 neigh- 
bours with s = — 1 and will refuse flipping to Sj = +1. 
In contrast , CA will first freeze all the links belonging to 
the tree, delete the (i,j) link and only then will perform 
a flip of the obtained macro in one painless step. 




Similarly, MRA will identify each end link and trans- 
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form its sites into a macro. Eventually, the algorithm will 
create a macro standing in for the entire tree. Clearly 
one can now update the macro of the new system in one 
step and then return to the explicit microscopic repre- 
sentation. This is similar to the DAMG procedure where 
the strongly connected degrees of freedom are iteratively 
connected into macros whose interactions are such as to 
represent correctly the energetics of the initial system. 



C. linear chains in gaussian distributions 



Consider a configuration of the type: 




The subsets A, B , C D, ("drops") can be considered 
as macros and minimized separately. Assume the block 
E minimization is preferring strongly a particular posi- 
tion for i and j. Take it for definitiveness s 8 - = + and 
Sj = + and assume that the product of the links joining 
i to j is -. The minimal energy configuration is there- 
fore frustrated. In order to find the minimum, one has 
to reach the configuration with only one unsatisfied link 
in the entire chain and to make such that the frustration 
resides on the link with the lowest |J| in the chain. 

As we mentioned in the previous section, CA is mainly 
freezing the strong satisfied links flips irreversibly the 
strong frustrated ones and keeps trying the weak ones. 
This is bringing very fast the configuration to the one 
with the frustration on the weakest link. 

MRA (similarly to AMG) is also putting together the 
satisfied strong links into macros. Once the entire ij line 
is transformed into just one macro, (with the strength of 
its lowest link) one can compare the price of its frustra- 
tion to the price of frustrating E (and decide which of 
the 2 should remain unsatisfied). 



D. Strongly coupled islands 

Consider a situation in which there are islands of sites 
related by very strong links submerged in a sea of links 
which are much weaker. 




The filled zones in the figure are the islands of strong 
links, and we did not draw their sites explicitly 4 . 

Suppose the strong links are of lower density and pos- 
sibly below the percolation . Suppose that the energy of 
the spins within the strong islands was somewhat min- 
imized. The equilibration of the relative signs between 
the the various islands is very inefficient by local updat- 
ing because none of the spins of an island would accept 
flipping without the entire island doing so. By defining 
each island as a macro, MRA can of course realign an 
entire island in one step and with quite high acceptance. 

CA is efficient too in as far as it freezes the strong 
links and it allows the others to reach an equilibrium 
(especially when annealing is applied). To check this, 
we considered a new model (the Strong- Weak model) in 
which the link distribution is given by: 

f(J) = lW-e) + 6(J + e)] 

+ + (7) 

As reported in Section 6 we found indeed for the 
Strong- Weak model a huge gap between the satisfied en- 
ergy per spin achieved by a CA and MRA on one side 
and by local Metropolis on the other side. 

MRA is more rapid than CA but in the case in which 
there are more than 2 typical energies for the links, the 
MRA is more difficult to apply while CA is still quite 
efficient in implicitly identifying macros in a stochastic 
manner. As seen in Section 6, this is especially true when 
one uses an annealed schedule, in which the temperature 



4 Beware! In many aspects, this figure might be quite misleading especially in the infinite dimensional case. 
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is gradually lowered. The temperature variation enable 
the CA to act on clusters at various scales, corresponding 
to the different temperatures and to address and freeze 
first the stronger links into small but very strongly cou- 
pled islands. At lower energies larger, loosely coupled 
islands are formed and acted upon. 



VI. COMPARING LOCAL MC TO MR A AND CA 



The Dilute SG model 

Simulations on the model defined by the equations 
(5) and (6) with a gaussian distribution for the links, 
/(J) oc exp(— J 2 ), were carried out comparatively using 
local dynamics (Metropolis), the Cluster Algorithm (CA) 
[25] and the Macros Reduction Algorithm (MRA). 

The simulations were carried out for various connectiv- 
ity values c and at temperatures below the glassy tran- 
sition T c . The size of the system was between 1000 and 
5000 sites. The results were averaged over at least 10 
different samples. A typical result is presented in Fig. 
1. In Fig. 1 one sees the evolution of the energy of the 
system (n=5000) monitored during its computer simula- 
tion (up to 50,000 MCS per spin). One can clearly see 
a gap between the energies reached with the global dy- 
namics (CA and MRA) and the local dynamics. MRA 
gets exactly the same energy level as the CA one but it 
converges much faster. 
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FIG. 1. E(t) the energy per spin as a function of the run- 
ning time for Metropolis , Cluster Algorithm (CA) and Macro 
Reduction Algorithm (RA) for a c = 2, N = 5000 lattice at 
T ~ 0.3T C . In the last 1000 MCS we put T = 0. 

In order to emphasize those features, we performed 
measurements using simulated annealing for the above 
dynamics. In Fig. 2 one can see similar picture of gap 
between the energies reached with the global dynamics 
(CA and MRA) and the local dynamics. 
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FIG. 2. Same as Fig. 
over the range T £ 2.0 — 



1 but with an annealing schedule 
0. 



This suggests that CA and MRA reach the true vac- 
uum while the local algorithm doesn't. 

In order to check directly this issue, we look next to 
the GS energy as found by global and local algorithms in 
a model for which we could estimate it analytically. 

The Strong-Weak Model 

The simulations of the Strong- Weak model (7) were 
compared with the analytical results in [26]. In [26] the 
self-consistent description of the low temperatures of the 
Strong- Weak, was based on the probability distribution 
of the local field defined by hi = Ttanh -1 < Si >t ■ 
Physically, this field is the first excitation, namely, in the 
limit T —> 0, \hi \ is the minimum energy cost for flipping 
the ith spin from its GS by the "best" reorganization 
of the system. This local field is in truth an oxymoron, 
since it depends on global properties of the cluster, the 
exchange field ^ - Jijirij, on the other hand, is truly a 
local property depending on the local connectivity (note 
that \hi\ < \ ^2j Jijrrij\). It was found that within the 
replica symmetry assumption the GS energy of the weak 
links is given by 

oo 

E W = --caP*e + -c(l - a)[£(l - iafc - h] (8) 



k = 



where a k = f + £,=i P h h = £~ _ M \h\P(h). Note 
that the energy of the strong links, Es = — \c(\ — a), is 
eliminated from eq. (8). 

The explicit value of Ew depends on the local field, 
P(h), which in general is difficult to calculate. Neverthe- 
less, after some work, P(h) can be determined from the 
equation 



P(h) = e 



-cQ 



^expHyH^(^ + e -«H 

Z7T Z 



S (l-a)£PK 



e iyU , e -iyUs 



(9) 



(=1 
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The resulting P(h) can then be compared with the usual 
random J = ±1 result Pi = exp(— cQ)I\\\ (cQ) [26] where 
Ii(x) is the modified Bessel function. The graphs of the 
two _P;'s are presented in the insert of Fig. 3. Note that 
after scaling e to 1, the Pi for the two cases are very 
close. However, the values of the exchange field are very 
different : 

• in the J = ±1 case the exchange field of a spin 
is usually not far from its local field (around the 
number of its neighbors), while 

• in the Weak-Strong model the local field is 0(e) vs. 
the exchange field which is 0(1). 

The fact that Pi is much smaller rises the hope that a 
global dynamics will be superior to the local one in the 
Strong- Weak model. 

This is confirmed by the following experiments: 
Performance of Local vs. Global Algorithms in the 
Strong-Weak model 

In the first set of runs we choose the connectivity c (the 
average number of neighbours) and the fraction of the 
strong links (1 — a), such that the density of the strong 
links by themselves is below the percolation threshold, 
cs = c(l — a) < 1. It is clear that in this situation all 
the strong links are unfrustrated, and the frustration is 
located only on the weak links. Therefore only the en- 
ergy of the weak links, Ew , is to be considered. In Fig. 
3 one can see a large steady difference (35%) in the en- 
ergy between the local dynamics and that of the cluster 
dynamics. 



In the second set of runs on the Weak-Strong model, 
we performed measurements using simulated annealing 
for both global and local dynamics (FIG 4.). This was 
performed by cooling over a wide range of temperatures. 
At larger temperatures, there were the strong links which 
reached their minimal energy and only then, at lower 
temperatures, the weak links adapted to the strong links 
environment. 
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FIG. 4. Ew(t) for the Weak-Strong model, c = 2 and 
a = 0.7. The annealing schedule range is T £ 123 — 0, with 
AT = 10 for T > 3 and AT = 0.1 for T < 3. The solid and 
the dotted lines indicate Cluster and Metropolis dynamics, 
respectively. The horizontal line denotes the analytical GS. 
Inset: Pi, for the WS case (solid), and J = ±1 case (dotted). 
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FIG. 3. Weak link energy Ew(i) as a function of the run- 
ning time. The model is Weak-Strong with c = 2, a = 0.7, 
N = 5000, T = 0.5 Jw- The strong and the weak links were 
scaled to 100 and 1, respectively. The solid and the dotted 
lines indicate Cluster and Metropolis dynamics, respectively. 
Inset: The first 50 steps. 

Strong-Weak with Simulated Annealing 



The clear energy difference between the local and clus- 
ter cases, is due to the fact that the local dynamics is 
totally stuck, since the probability of flipping a cluster 
consisting of strong links is practically zero for the local 
algorithm. On the other hand, the CA "knows" how to 
deal with the strong link structures by treating them as 
only "one degree of freedom" , for each cluster. In other 
words, the CA is extremely efficient in the solving the 
problem: "How to arrange the weak links in the environ- 
ment of the strong links" . 

RA gets the same results faster by defining the islands 
explicitly as macros and and manipulating them directly. 

The type of the mean field solution for such models 
[26] is known to be unstable [22]. However, in Fig. 4 
one can see that the analytical GS [26], is in very good 
agreement with the averaged GS energy obtained by the 
CA. 



VII. OUTLOOK 

Identifying the nature and the dynamics of the macros 
may help understand the complex multiscale nature of 
a system. The techniques for identifying the relevant 
macros lead to a deeper understanding of the way the 
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macroscopic description of our world arises in the study 
of complex systems composed of simple microscopic ele- 
ments. 

The macros can be multiscale reducible but in many 
cases there might exist complex irreducible cores. While 
such irreducible macros might have fortuitous character- 
istics, lack generality and present non-generic properties, 
they might be very important if the same set of cores ap- 
pears recurrently in biological, neurological or cognitive 
systems in nature. 

In such situations, rather than trying to understand 
the macros structure, dynamics and properties on gen- 
eral (multiscale, analytic) grounds as collections of their 
parts, one may have to recognize the unity and unique- 
ness of these macros and resign oneself in just making an 
as intimate as possible acquaintance with their features. 

One may still try to treat them by the implicit elim- 
ination method [12] where the complex objects are pre- 
senting, isolating and eliminating themselves by the very 
fact that they are projected out by the dynamics as the 
slow-to-converge modes. 
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APPENDIX 

We present in this appendix the proofs of the theorems. 

A. THEOREM I 

Consider 2 subsets A and B of an ultrametric (UM) 
system S which has a GS (ground state) S° . Assume 
that the states S A and S B obtained by flipping the spins 
of the set A (respectively B) in S° are GS's too. 

Then 

#(Af]B)>l/2min(#A,#B) (10) 

Namely, the smaller of the sets A and B share at least 
half of itself with the big one. Moreover, if 

#A>#B 

then 
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#(Af]B) = l/2#B. (11) 

PROOF: 

1. d(S A ,S°) = #A, d(S B ,S°) = #B and 
d(S B ,S A ) = #A + #B- 2#(A B) 

2. On the other hand UM implies < 
max[d(S i ,S k ),d(S k ,S j )], Vi,j,k. Namely, 

(a) either all the distances are equal: d(S\S 3 ) = 
d(S\S k ) = d(S k ,Si) or, 

(b) two distances are equal and the third is shorter 
(e.g.): d(5*',5 J ') < d(S i ,S k ) = d(S k ,S j ) 

3. Considering (1) and (2) for S° , S A and S B , one 
finds with some simple arithmetic that: 

(a) in the first case (#A ^ #5): 

#(Af]B) = l/2min(#A,#B) 

(b) in the second case (#j4 = #B): 

#(Af]B) > 1/2#A 

□ 

Definition 

Let A be a subset of the system S. We say A , a con- 
figuration of A is locally optimized if A is a minimum 
of the internal energy of A ignoring the energy due to the 
interaction with the rest of S (which we denote by A). 

B. THEOREM II 

If S° and S A differing by the sign of the spins in the 
region A are both GS of a (not necessarily ultrametric) 
system S. 

Denote by A the restriction of the configuration S° to 

A. 

Then A is "locally optimized". 
PROOF: 

Assume that A is not locally optimized therefore 
there exists a sub-set A* C A which can be flipped to 
bring the system A to the configurations A 2 , which is a 
GS of A. Alternatively one can flip A* (the complement 
of A* in A) to bring the system A to a configurations 
A 3 , which is a GS of A as well. 

The 2 states A 2 and A 3 differ by the flip of the entire A. 
Define the configuration S 2 which is S° when restricted 
to A and A 2 when restricted to A. Similarly define S 3 . 



The energy differences ignoring the interaction be- 
tween A and A are: 

e[A°] - e[A 2 ] = e[A°] - e[A 3 ] > 0(1), and positive. 
The overall energies are: 

E[S 2 } = e[A 2 } + e[A°] + E[A 2 ® A ] 
E[S 3 ] = e[A 3 ] + e[A°] + E[A 3 ® A ] 

E[S°] = e[A°] + e[A°] + E[A° ® A ] 

where £ , [ j 4 2 (8)j4°] is the energy of the interaction between 
A 2 and A etc. Using those facts together with the fact 
that E[A 2 (g) A ] = —E[A 3 <g> A ], it is easy to see that: 
Either 

• E[S°] - E[S 2 } > 0(1) and positive. 
Or 

• E[S°] - E[S 3 ] > 0(1) and positive. 

This means that at least one of the states S 2 or S 3 
have lower energy than S° (by more than 0(1). In turn, 
this contradicts the initial given that S° is a GS. There- 
fore our assumption (that A is not locally optimized) 
must be wrong, and A is locally optimized. 
□ 



C. THEOREM III 

Following the definitions of theorem II we argue: 

If A is locally optimized, in the GS S A , and S has an 
ultrametric structure. Then the restriction of S A to A is 
the unique GS of A (up to global flipping of A). 

PROOF: 

Following the definitions and operations at of theorem 
II, if the system has another GS, there exists A* which 
can be flipped to bring the system A to the configura- 
tions A 2 , which is a another GS of A. Alternatively one 
can flip A* to bring the system A to the configurations 
A 3 , which is a GS of A as well. 
Namely: 

• e[A] - e[A 2 ] = e[A] - e[A 3 ] = 0(1) and positive. 
Using the similar reasoning as for the prove of theorem II: 

• The overall energy is E[S A ] = e[A 2 ]+e[A 2 ]+E[A 2 <g> 
A 2 ] 

• E[A 2 <g) A 2 } = 0(1) 

• This imply that E[A* 2 <g> A 2 ] = 0(1) and E[A* 2 <g> 
A 2 ] = 0(1), otherwise, one of the configuration S 2 
or S 3 has a lower energy (> 0(1) than S°). 
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One obtains: 

• both E[S°] - E[S 2 ] = 0(1) and E[S°] - E[S 3 ] = 0(1) 

Therefore A* and A* are two separated objects which 
by flipping can bring the GS S° to other 2 GS's. But 
this violates theorem I. Therefore the assumption that A 



may have more than one ground state is false. □ 

Note that the results are independent from the defini- 
tion of what is a GS. For instance in theorem II, if one 
would be interested in "GS-a" -states defined by the re- 
lation E S o - EgA < 0{N 1 l a ) (where 5° is a GS). The 
theorem would be still valid in terms of "a-optimization" . 
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